Method and apparatus for determining vehicle class based upon audio data

ABSTRACT

A method, apparatus and computer program product are provided to identify the class of vehicle driving over a road surface based upon audio data collected as the vehicle drives thereover. With respect to predicting a class of a vehicle, audio data is obtained that is created by the vehicle while driving over the road surface. The audio data includes one or more audio frequency features and/or one or more audio amplitude features. The audio data including the one or more audio frequency features and/or the one or more audio amplitude features is provided to a machine learning model and the class of the vehicle that created the audio data is predicted utilizing the machine learning model. A method, apparatus and computer program product are also provided for training the machine learning model to predict the class of the vehicle driving over the road surface.

CROSS REFERENCE TO RELATED APPLICATION

This application claims benefit of U.S. Provisional Application No.63/031,036, filed May 28, 2020, which is incorporated herein byreference in its entirety.

TECHNOLOGICAL FIELD

An example embodiment relates generally to a method, apparatus andcomputer program product for determining the class of a vehicle and,more particularly, for determining the class of a vehicle based uponaudio data generated by the vehicle driving over a road surface.

BACKGROUND

Modern vehicles include a plurality of different types of sensors forcollecting a wide variety of information. These sensors include locationsensors, such as global positioning system (GPS) sensors, configured todetermine the location of the vehicle. Based upon the location of thevehicle, a variety of navigational, mapping and other services may beprovided for manually driven vehicles as well as the provision ofnavigation and control of autonomous or semi-autonomous vehicles. Otherexamples of sensors include cameras or other imaging sensors thatcapture images of the environment including objects in the vicinity ofthe vehicle. The images that are captured may be utilized to determinethe location of the vehicle with more precision. A more precisedetermination of the vehicle location may be useful in conjunction withthe provision of navigational, mapping and other informational servicesfor a manually driven vehicle. Additionally, the more precisedetermination of the vehicle location may provide for the improvednavigation and control of an autonomous or semi-autonomous vehicle bytaking into account the location of other objects, such as othervehicles, in proximity to the vehicle carrying the sensors.

The sensors on board vehicles therefore collect a wide variety of datathat may be utilized for various purposes. However, these sensorscurrently on board vehicles do have limitations and do not collect allof the different types of information that would be useful in variousapplications.

BRIEF SUMMARY

A method, apparatus and computer program product are provided inaccordance with an example embodiment in order to identify the class ofvehicle driving over a road surface. In this regard, the method,apparatus and computer program product of an example embodiment utilizeaudio data collected as the vehicle drives over the road surface andidentifies the class of the vehicle based upon the audio data. Thereliance upon the collection and analysis of audio data may supplementthe information provided by other sensors on board the vehicle and allowfor the provision of different information, such as the class of thevehicle, that is useful for a variety of applications. By way ofexample, the identification of the class of vehicle driving over a roadsurface may be useful for federal, state, local or other governmental orregulatory officials that design and maintain the road network. Asanother example, the identification of the class of vehicle driving overa road surface may be useful in relation to the provision of morerelevant information to the vehicle and/or the driver of the vehicle.

In an example embodiment, an apparatus is provided that is configured topredict a class of a vehicle based at least in part on audio data. Theapparatus includes at least one processor and at least one memorystoring computer program code with the at least one memory and thecomputer program code configured to, with the processor, cause theapparatus to at least obtain the audio data created by the vehicle whiledriving over a road surface. The audio data includes a plurality ofaudio features including one or more audio amplitude features and/or oneor more audio frequency features. The at least one memory and thecomputer program code are also configured to, with the processor, causethe apparatus to provide the audio data including the one or more audioamplitude features and/or the one or more audio frequency features to amachine learning model. The at least one memory and the computer programcode are further configured to, with the processor, cause the apparatusto predict the class of the vehicle that created the audio datautilizing the machine learning model.

The at least one memory and the computer program code are furtherconfigured to, with the processor, cause the apparatus of an exampleembodiment to identify one or more map objects and to provideinformation regarding the one or more map objects to the machinelearning model. The at least one memory and the computer program codeare also configured to, with the processor, cause the apparatus of thisexample embodiment to predict the class of the vehicle by predicting theclass of the vehicle utilizing the machine learning model and based uponthe one or more map objects and the audio data. In this exampleembodiment, the at least one memory and the computer program code may befurther configured to, with the processor, cause the apparatus toreceive an indication of a location of the vehicle. The at least onememory and the computer program code are also configured to, with theprocessor, cause the apparatus of this example embodiment to obtain theaudio data by obtaining the audio data captured proximate the locationof the vehicle. The at least one memory and the computer program codeare further configured to, with the processor, cause the apparatus ofthis example embodiment to identify the one or more map objects byidentifying the one or more map objects proximate the location of thevehicle.

The at least one memory and the computer program code are configured to,with the processor, cause the apparatus of an example embodiment toobtain the audio data by capturing the audio data associated with one ormore respective wheels of the vehicle while driving over the roadsurface and associating the location of the vehicle during creation ofthe audio data therewith. In an example embodiment, the one or moreaudio amplitude features represent a change in amplitude relative to anoise floor and/or the one or more audio frequency features represent achange in frequency relative to a noise floor. The at least one memoryand the computer program code are further configured to, with theprocessor, cause the apparatus of this example embodiment to receive anindication of a speed of the vehicle and to determine the noise floorbased at least in part upon the speed of the vehicle.

In another example embodiment, a method is provided for predicting aclass of a vehicle based at least in part on audio data. The methodincludes obtaining the audio data created by the vehicle while drivingover a road surface. The audio data includes a plurality of audiofeatures including one or more audio frequency features and/or one ormore audio amplitude features. The method also includes providing theaudio data including the one or more audio frequency features and/or theone or more audio amplitude features to a machine learning model andpredicting the class of the vehicle that created the audio datautilizing the machine learning model.

The method of an example embodiment also includes identifying one ormore map objects and providing information regarding the one or more mapobjects to the machine learning model. In this example embodiment, themethod predicts the class of the vehicle by predicting the class of thevehicle utilizing the machine learning model and based upon the one ormore map objects and the audio data. The method of this exampleembodiment may also include receiving an indication of a location of thevehicle. In this example embodiment, the method obtains the audio databy obtaining the audio data captured proximate the location of thevehicle, and identifies the one or more map objects by identifying theone or more map objects proximate the location of the vehicle.

The method of an example embodiment obtains the audio data by capturingthe audio data associated with one or more respective wheels of thevehicle while driving over the road surface and associates the locationof the vehicle during creation of the audio data therewith. In anexample embodiment, the one or more audio amplitude features represent achange in amplitude relative to a noise floor and/or the one or moreaudio frequency features represent a change in frequency relative to anoise floor. The method of this example embodiment also includesreceiving an indication of a speed of the vehicle and determining thenoise floor based at least in part upon the speed of the vehicle.

In a further example embodiment, a computer program product is providedthat includes a non-transitory computer readable medium having programcode portions stored thereon with the program code portions beingconfigured, upon execution, to predict a class of a vehicle based atleast in part on audio data. The program code portions are alsoconfigured to obtain the audio data created by the vehicle while drivingover a road surface. The audio data includes a plurality of audiofeatures including one or more audio frequency features and/or one ormore audio amplitude features. The program code portions are furtherconfigured to provide the audio data including the one or more audiofrequency features and/or the one or more audio amplitude features to amachine learning model and to predict the class of the vehicle thatcreated the audio data utilizing the machine learning model.

The program code portions of an example embodiment are also configuredto identify one or more map objects and to provide information regardingthe one or more map objects to the machine learning model. In thisexample embodiment, the program code portions configured to predict theclass of the vehicle include program code portions are configured topredict the class of the vehicle utilizing the machine learning modeland based upon the one or more map objects and the audio data. Theprogram code portions of this example embodiment may also be configuredto receive an indication of a location of the vehicle. In this exampleembodiment, the program code portions configured to obtain the audiodata include program code portions configured to obtain the audio datacaptured proximate the location of the vehicle, and the program codeportions configured to identify the one or more map objects includeprogram code portions configured to identify the one or more map objectsproximate the location of the vehicle.

The program code portions of an example embodiment that are configuredto obtain the audio data include program code portions configured tocapture the audio data associated with one or more respective wheels ofthe vehicle while driving over the road surface and program codeportions configured to associate the location of the vehicle duringcreation of the audio data therewith. In an example embodiment, the oneor more audio amplitude features represent a change in amplituderelative to a noise floor and/or the one or more audio frequencyfeatures represent a change in frequency relative to a noise floor. Theprogram code portions of this example embodiment also include programcode portions configured to receive an indication of a speed of thevehicle and program code portions configured to determine the noisefloor based at least in part upon the speed of the vehicle.

In yet another example embodiment, an apparatus is provided forpredicting a class of a vehicle based at least in part on audio data.The apparatus includes means for obtaining the audio data created by thevehicle while driving over a road surface. The audio data includes aplurality of audio features including one or more audio frequencyfeatures and/or one or more audio amplitude features. The apparatus alsoincludes means for providing the audio data including the one or moreaudio frequency features and/or the one or more audio amplitude featuresto a machine learning model and means for predicting the class of thevehicle that created the audio data utilizing the machine learningmodel.

The apparatus of an example embodiment also includes means foridentifying one or more map objects and means for providing informationregarding the one or more map objects to the machine learning model. Inthis example embodiment, the means for predicting the class of thevehicle includes means for predicting the class of the vehicle utilizingthe machine learning model and based upon the one or more map objectsand the audio data. The apparatus of this example embodiment may alsoinclude means for receiving an indication of a location of the vehicle.In this example embodiment, the means for obtaining the audio dataincludes means for obtaining the audio data captured proximate thelocation of the vehicle, and the means for identifying the one or moremap objects includes means for identifying the one or more map objectsproximate the location of the vehicle.

In an example embodiment, the means for obtaining the audio dataincludes means for capturing the audio data associated with one or morerespective wheels of the vehicle while driving over the road surface andmeans for associating the location of the vehicle during creation of theaudio data therewith. In an example embodiment, the one or more audioamplitude features represent a change in amplitude relative to a noisefloor and/or the one or more audio frequency features represent a changein frequency relative to a noise floor. The apparatus of this exampleembodiment also includes means for receiving an indication of a speed ofthe vehicle and means for determining the noise floor based at least inpart upon the speed of the vehicle.

In an example embodiment, an apparatus is provided that is configured totrain a machine learning model to predict a class of an unknown vehicledriving over a road surface. The apparatus includes at least oneprocessor and at least one memory storing computer program code with theat least one memory and the computer program code configured to, withthe processor, cause the apparatus to at least provide a trainingdataset including a plurality of training examples. A respectivetraining example includes audio data created by a vehicle of a knownclass while driving over a road surface. The audio data includes aplurality of audio features including one or more audio featuresselected from a group of one or more audio amplitude features and/or oneor more audio frequency features. The respective training example alsoincludes the class of the vehicle that created the audio data. The atleast one memory and the computer program code are also configured to,with the processor, cause the apparatus to train the machine learningmodel utilizing the training dataset such that the machine learningmodel, as trained, is configured to predict the class of the unknownvehicle driven over the road surface based at least in part upon theaudio data created by the unknown vehicle.

The respective training example may also include a map object. In thisexample embodiment, the at least one memory and the computer programcode are also configured to, with the processor, cause the apparatus toprovide the training dataset by constructing the respective trainingexample by identifying the map object by reference to a map database andidentifying the audio data proximate the map object by reference to anaudio database. In this example embodiment, the audio data is associatedwith a location proximate the map object at which the audio data waspreviously captured. The at least one memory and the computer programcode may be further configured to, with the processor, cause theapparatus of this example embodiment to construct the respectivetraining example by identifying the class of the vehicle that createdthe audio data while driving over the road surface at the locationproximate the map object. In this example embodiment, the at least onememory and the computer program code may also be configured to, with theprocessor, cause the apparatus to identify the audio data by identifyingthe audio data proximate the map object from an audio layer of the mapdatabase. The audio layer of this example embodiment defines a pluralityof instances of audio data in association with respective locations. Therespective training example may additionally include a speed of thevehicle while driving over the road surface.

In another example embodiment, a method is provided that is configuredto train a machine learning model to predict a class of an unknownvehicle driving over a road surface. The method includes providing atraining dataset including a plurality of training examples. Arespective training example includes audio data created by a vehicle ofa known class while driving over a road surface. The audio data includesa plurality of audio features including one or more audio featuresselected from a group of one or more audio amplitude features and/or oneor more audio frequency features. The respective training example alsoincludes the class of the vehicle that created the audio data. Themethod also includes training the machine learning model utilizing thetraining dataset such that the machine learning model, as trained, isconfigured to predict the class of the unknown vehicle driven over theroad surface based at least in part upon the audio data created by theunknown vehicle.

The respective training example may also include a map object. In thisexample embodiment, the method provides the training dataset byconstructing the respective training example by identifying the mapobject by reference to a map database and identifying the audio dataproximate the map object by reference to an audio database. In thisexample embodiment, the audio data is associated with a locationproximate the map object at which the audio data was previouslycaptured. The method may construct the respective training example byidentifying the class of the vehicle that created the audio data whiledriving over the road surface at the location proximate the map object.In this example embodiment, the method may identify the audio data byidentifying the audio data proximate the map object from an audio layerof the map database. The audio layer of this example embodiment definesa plurality of instances of audio data in association with respectivelocations. The respective training example may additionally include aspeed of the vehicle while driving over the road surface.

In a further example embodiment, a computer program product is providedthat is configured to train a machine learning model to predict a classof an unknown vehicle driving over a road surface. The computer programproduct includes a non-transitory computer readable medium havingprogram code portions stored thereon with the program code portionsbeing configured, upon execution, to provide a training datasetincluding a plurality of training examples. A respective trainingexample includes audio data created by a vehicle of a known class whiledriving over a road surface. The audio data includes a plurality ofaudio features including one or more audio features selected from agroup of one or more audio amplitude features and/or one or more audiofrequency features. The respective training example also includes theclass of the vehicle that created the audio data. The program codeportions are also configured to train the machine learning modelutilizing the training dataset such that the machine learning model, astrained, is configured to predict the class of the unknown vehicledriven over the road surface based at least in part upon the audio datacreated by the unknown vehicle.

The respective training example may also include a map object. In thisexample embodiment, the program code portions configured to provide thetraining dataset include program code portions configured to constructthe respective training example by identifying the map object byreference to a map database and identifying the audio data proximate themap object by reference to an audio database. In this exampleembodiment, the audio data is associated with a location proximate themap object at which the audio data was previously captured. The programcode portions configured to construct the respective training examplemay include program code portions configured to identify the class ofthe vehicle that created the audio data while driving over the roadsurface at the location proximate the map object. In this exampleembodiment, the program code portions configured to identify the audiodata include program code portions configured to identify the audio dataproximate the map object from an audio layer of the map database. Theaudio layer of this example embodiment defines a plurality of instancesof audio data in association with respective locations. The respectivetraining example may additionally include a speed of the vehicle whiledriving over the road surface.

In yet another example embodiment, an apparatus is provided that isconfigured to train a machine learning model to predict a class of anunknown vehicle driving over a road surface. The apparatus includesmeans for providing a training dataset including a plurality of trainingexamples. A respective training example includes audio data created by avehicle of an unknown class while driving over a road surface. The audiodata includes a plurality of audio features including one or more audiofeatures selected from a group of one or more audio amplitude featuresand/or one or more audio frequency features. The respective trainingexample also includes the class of the vehicle that created the audiodata. The apparatus also includes means for training the machinelearning model utilizing the training dataset such that the machinelearning model, as trained, is configured to predict the class of theunknown vehicle driven over the road surface based at least in part uponthe audio data created by the unknown vehicle.

The respective training example may also include a map object. In thisexample embodiment, the means for providing the training datasetincludes means for constructing the respective training example byidentifying the map object by reference to a map database andidentifying the audio data proximate the map object by reference to anaudio database. In this example embodiment, the audio data is associatedwith a location proximate the map object at which the audio data waspreviously captured. The means for constructing the respective trainingexample may include means for identifying the class of the vehicle thatcreated the audio data while driving over the road surface at thelocation proximate the map object. In this example embodiment, the meansfor identifying the audio data includes means for identifying the audiodata proximate the map object from an audio layer of the map database.The audio layer of this example embodiment defines a plurality ofinstances of audio data in association with respective locations. Therespective training example may additionally include a speed of thevehicle while driving over the road surface.

BRIEF DESCRIPTION OF THE DRAWINGS

Having thus described certain embodiments of the invention in generalterms, reference will now be made to the accompanying drawings, whichare not necessarily drawn to scale, and wherein:

FIG. 1 is a block diagram of an apparatus that may be specificallyconfigured in accordance with an example embodiment;

FIG. 2 is a flowchart illustrating the operations performed, such as bythe apparatus of FIG. 1, in order to train a machine learning model toidentify the class of a vehicle based upon audio data generated by thevehicle driving over a road surface in accordance with an exampleembodiment;

FIG. 3 is a flowchart illustrating the operations performed, such as bythe apparatus of FIG. 1, in order to construct a training example inaccordance with an example embodiment;

FIG. 4 is a graphical representation of a road surface having severaldifferent map objects, such as a set of railroad tracks and a seamdefining a transition to a different road surface, such as the roadsurface of a bridge or the like, as well as audio data associated withdifferent locations proximate the various map objects and created byvehicles driving over the road surface; and

FIG. 5 is a flowchart illustrating the operations performed, such as bythe apparatus of FIG. 1, in accordance with an example embodiment inorder to utilize a trained machine learning model to identify the classof a vehicle driving over a road surface based upon audio data.

DETAILED DESCRIPTION

Some embodiments of the present invention will now be described morefully hereinafter with reference to the accompanying drawings, in whichsome, but not all, embodiments of the invention are shown. Indeed,various embodiments of the invention may be embodied in many differentforms and should not be construed as limited to the embodiments setforth herein; rather, these embodiments are provided so that thisdisclosure will satisfy applicable legal requirements. Like referencenumerals refer to like elements throughout. As used herein, the terms“data,” “content,” “information,” and similar terms may be usedinterchangeably to refer to data capable of being transmitted, receivedand/or stored in accordance with embodiments of the present invention.Thus, use of any such terms should not be taken to limit the spirit andscope of embodiments of the present invention.

A method, apparatus and computer program product are provided inaccordance with an example embodiment in order to determine the class ofa vehicle driving over a road surface based upon audio data generated bythe vehicle driving over the road surface and interacting, for example,with one or more objects along the road. By relying upon audio data, aricher and more complete set of data characterizing the vehicle and theroad surface may be collected with the information derived from theaudio data, such as the class of vehicle, being useful for a variety ofdifferent applications. In order to determine the class of vehicle basedupon audio data, the method, apparatus and computer program product ofan example embodiment are configured to train a machine learning modelutilizing training examples that include the class of vehicle and audiodata generated by the vehicle while driving over a road surface. Oncetrained, the method, apparatus and computer program product of anotherexample embodiment utilize the trained machine learning model to analyzeaudio data that is collected as a vehicle drives over a road surface andthat then determines the class of the vehicle that generated the audiodata.

The apparatus that is configured to train a machine learning model toidentify the class of a vehicle based upon audio data generated by thevehicle while driving over a road surface and the apparatus that isconfigured to utilize a trained machine learning model to determine thevehicle class based upon audio data generated by the vehicle whiledriving over a road surface may each be any of a wide variety ofcomputing devices and may be embodied by either the same or differentcomputing devices. With respect to the computing device configured totrain a machine learning model to identify the class of a vehicle basedupon audio data generated by the vehicle while driving over a roadsurface, the apparatus may be embodied by a server, a computerworkstation, a distributed network of computing devices, a personalcomputer or any other type of computing device, while the apparatusconfigured to utilize a trained machine learning model to identify theclass of vehicle based upon the audio data generated by the vehiclewhile driving over a road surface may similarly be embodied by the sameor different server, computer workstation, distributed network ofcomputing devices, personal computer or other type of computing device.

Alternatively, the apparatus configured to utilize the trained machinelearning model to determine the class of a vehicle based upon the audiodata generated by the vehicle while driving over a road surface may beembodied by a computing device on board the vehicle, such as a computersystem of a vehicle, e.g., a computing device of a vehicle that supportssafety-critical systems such as the powertrain (engine, transmission,electric drive motors, etc.), steering (e.g., steering assist orsteer-by-wire), and/or braking (e.g., brake assist or brake-by-wire), anavigation system of a vehicle, a control system of a vehicle, anelectronic control unit of a vehicle, an autonomous vehicle controlsystem (e.g., an autonomous-driving control system) of a vehicle, amapping system of a vehicle, an Advanced Driver Assistance System (ADAS)of a vehicle), or any other type of computing device carried by thevehicle. Still further, the apparatus configured to utilize the trainedmachine learning model to determine the class of a vehicle based uponthe audio data generated by the vehicle while driving over a roadsurface may be embodied by a computing device of a driver or passengeron board the vehicle, such as a mobile terminal, e.g., a personaldigital assistant (PDA), mobile telephone, smart phone, personalnavigation device, smart watch, tablet computer, or any combination ofthe aforementioned and other types of portable computer devices.

Regardless of the manner in which the apparatus is embodied, however,the apparatus 10 includes, is associated with, or is in communicationwith processing circuitry 12, memory 14, a communication interface 16and optionally a user interface 18 as shown in FIG. 1. In someembodiments, the processing circuitry (and/or co-processors or any otherprocessors assisting or otherwise associated with the processingcircuitry) can be in communication with the memory via a bus for passinginformation among components of the apparatus. The memory can benon-transitory and can include, for example, one or more volatile and/ornon-volatile memories. In other words, for example, the memory may be anelectronic storage device (for example, a computer readable storagemedium) comprising gates configured to store data (for example, bits)that can be retrievable by a machine (for example, a computing devicelike the processing circuitry). The memory can be configured to storeinformation, data, content, applications, instructions, or the like forenabling the apparatus to carry out various functions in accordance withan example embodiment of the present disclosure. For example, the memorycan be configured to buffer input data for processing by the processingcircuitry. Additionally or alternatively, the memory can be configuredto store instructions for execution by the processing circuitry.

The processing circuitry 12 can be embodied in a number of differentways. For example, the processing circuitry may be embodied as one ormore of various hardware processing means such as a processor, acoprocessor, a microprocessor, a controller, a digital signal processor(DSP), a processing element with or without an accompanying DSP, orvarious other processing circuitry including integrated circuits suchas, for example, an ASIC (application specific integrated circuit), anFPGA (field programmable gate array), a microcontroller unit (MCU), ahardware accelerator, a special-purpose computer chip, or the like. Assuch, in some embodiments, the processing circuitry can include one ormore processing cores configured to perform independently. A multi-coreprocessor can enable multiprocessing within a single physical package.Additionally or alternatively, the processing circuitry can include oneor more processors configured in tandem via the bus to enableindependent execution of instructions, pipelining and/or multithreading.

In an example embodiment, the processing circuitry 12 can be configuredto execute instructions stored in the memory 14 or otherwise accessibleto the processing circuitry. Alternatively or additionally, theprocessing circuitry can be configured to execute hard codedfunctionality. As such, whether configured by hardware or softwaremethods, or by a combination thereof, the processing circuitry canrepresent an entity (for example, physically embodied in circuitry)capable of performing operations according to an embodiment of thepresent disclosure while configured accordingly. Thus, for example, whenthe processing circuitry is embodied as an ASIC, FPGA or the like, theprocessing circuitry can be specifically configured hardware forconducting the operations described herein. Alternatively, as anotherexample, when the processing circuitry is embodied as an executor ofsoftware instructions, the instructions can specifically configure theprocessing circuitry to perform the algorithms and/or operationsdescribed herein when the instructions are executed. However, in somecases, the processing circuitry can be a processor of a specific device(for example, a computing device) configured to employ an embodiment ofthe present disclosure by further configuration of the processor byinstructions for performing the algorithms and/or operations describedherein. The processing circuitry can include, among other things, aclock, an arithmetic logic unit (ALU) and/or one or more logic gatesconfigured to support operation of the processing circuitry.

The apparatus 10 of an example embodiment can also include thecommunication interface 16 that can be any means such as a device orcircuitry embodied in either hardware or a combination of hardware andsoftware that is configured to receive and/or transmit data from/toother electronic devices in communication with the apparatus, such as adatabase 24 which, in one embodiment, comprises a map database thatstores data (e.g., one or more map objects, audio data, etc.) generatedand/or employed by the processing circuitry 12. Additionally oralternatively, the communication interface can be configured tocommunicate in accordance with various wireless protocols includingGlobal System for Mobile Communications (GSM), such as but not limitedto Long Term Evolution (LTE). In this regard, the communicationinterface can include, for example, an antenna (or multiple antennas)and supporting hardware and/or software for enabling communications witha wireless communication network. In this regard, the communicationinterface can include, for example, an antenna (or multiple antennas)and supporting hardware and/or software for enabling communications witha wireless communication network. Additionally or alternatively, thecommunication interface can include the circuitry for interacting withthe antenna(s) to cause transmission of signals via the antenna(s) or tohandle receipt of signals received via the antenna(s). In someenvironments, the communication interface can alternatively or alsosupport wired communication and/or may alternatively support vehicle tovehicle or vehicle to infrastructure wireless links.

In certain embodiments, the apparatus 10 can be equipped or associatedwith one or more positioning sensors 20, such as one or more GPSsensors, one or more accelerometer sensors, one or more light detectionand ranging (LiDAR) sensors, one or more radar sensors, one or moregyroscope sensors, and/or one or more other sensors. Any of the one ormore sensors may be used to sense information regarding movement,positioning and location, and/or orientation of the apparatus for use,such as by the processing circuitry 12, in navigation assistance and/orautonomous vehicle control, as described herein according to exampleembodiments.

In certain embodiments, the apparatus 10 may further be equipped with orin communication with one or more audio sensors 22. In an example,embodiment, the one or more audio sensors can be one or more audiocapturing devices (e.g., one or more microphones). In some exampleembodiments, the one or more audio sensors can be implemented in avehicle or other remote apparatus. For example, the one or more audiosensors can be located proximate to one or more wheels of a vehicle.While embodiments may be implemented with a single microphone audiosensor, particularly if that audio sensor is capable of directionalaudio detection and positioned to capture audio from multiple points(e.g., from each wheel), other embodiments may include microphone audiosensors at each wheel to capture sounds related to a tire of each wheelas it contacts the roadway. Microphones may be placed inside the body ofa wheel directly, between the wheel rim and tire, or the wheel wells ofa vehicle to shield the microphone from ambient noises while alsoshielding each microphone from at least some of the noise generated byother wheels around the vehicle. A microphone may optionally be placeddirectly above a wheel as tires are generally hollow, semi-rigidsurfaces that may effectively transfer the sound through the tire to amicrophone above the tire. Certain embodiments may embed microphonesdirectly into the wheel attached to a tire rim, such as within the tirebetween the tire and the rim similar to conventional tire pressuremonitoring sensors. Further, the audio sensors may be directionalmicrophones that may focus their attention at the contact patch wherethe respective tire meets the road to help further isolate the roadnoise from each respective tire from surrounding sounds.

The locations of the wheels may each be identified in latitude andlongitude based on a location of the vehicle using a sensor, such as aGPS sensor to identify the location of the vehicle. Each vehicle or typeof vehicle (e.g., make and model) may also be uniquely configured toidentify an offset between the GPS sensor and each of the wheels. Thus,understanding a position of the vehicle through a position of GPS sensorprovides an identification of the location of the wheel using theoffsets.

In certain embodiments, information detected by the one or more audiosensors can be transmitted to the apparatus 10, such as the processingcircuitry 12, as audio data. The audio data transmitted by the one ormore audio sensors can be transmitted via one or more wiredcommunications and/or one or more wireless communications (e.g., nearfield communication, or the like). In some environments, thecommunication interface 16 can support wired communication and/orwireless communication with the one or more audio sensors.

The apparatus 10 may also optionally include a user interface 18 thatmay, in turn, be in communication with the processing circuitry 12 toprovide output to the user and, in some embodiments, to receive anindication of a user input. As such, the user interface may include adisplay and, in some embodiments, may also include a keyboard, a mouse,a joystick, a touch screen, touch areas, soft keys, one or moremicrophones, a plurality of speakers, or other input/output mechanisms.In one embodiment, the processing circuitry may comprise user interfacecircuitry configured to control at least some functions of one or moreuser interface elements such as a display and, in some embodiments, aplurality of speakers, a ringer, one or more microphones and/or thelike. The processing circuitry and/or user interface circuitry embodiedby the processing circuitry may be configured to control one or morefunctions of one or more user interface elements through computerprogram instructions (for example, software and/or firmware) stored on amemory accessible to the processing circuitry (for example, memory 14,and/or the like).

Referring now to FIG. 2, the operations performed, such as by theapparatus 10 of FIG. 1, in order to train a machine learning model todetermine the class of an unknown vehicle based upon audio datagenerated by the vehicle while driving over a road surface are depicted.As used herein, a vehicle that subsequently drives over the roadsurface, that is, that drives over the road surface after the machinelearning model has been trained, is unknown in that the class of thevehicle is not known in advance or at least not known with a desireddegree of certainty. As shown in block 30, the apparatus includes means,such as the processing circuitry 12, memory 14, the communicationinterface 16 or the like, for providing a training data set thatincludes a plurality of training examples. In this regard, the trainingdata set may be provided by access by the processing circuitry of thetraining data set stored by the memory. Alternatively, the training dataset may be provided by access by the processing circuitry to a database24 or other memory device that either may be a component of theapparatus or may be separate from, but accessible to the apparatus, suchas the processing circuitry, via the communication interface.

A training example includes audio data created by a vehicle of a knownclass while driving over a road surface. The audio data includes one ormore audio features including one or more audio features selected from agroup of one or more audio amplitude features and/or one or more audiofrequency features. As such, the audio data may include one or moreaudio amplitude features without any audio frequency features, one ormore audio frequency features without any audio amplitude features, orone or more audio amplitude features as well as one or more audiofrequency features. In some embodiments, the audio data may includeaudio features that are based on a combination, e.g., correlation, ofone or more audio amplitude features and one or more audio frequencysignals. The audio data that comprises a training example, including theone or more audio features, is collected at the same point in time.Thus, the audio amplitude feature(s) and/or the audio frequencyfeature(s) of a training example have each been collected at the sametime in response to the vehicle driving over a road surface. However,different training examples may include audio data collected atdifferent points in time.

The training example also includes the class of the vehicle that createdthe audio data, such as a label identifying the class of vehicle. Inthis regard, the class of vehicle defines the type of vehicle and whilevehicle class may be differently defined in various embodiments and bydifferent countries, the Federal Highway Administration of the UnitedStates has adopted definitions of a plurality of classes of vehiclesincluding motorcycles, passenger cars, other two-axle four-tire singleunit vehicles such as pickups and vans, buses, two-axle six-tiresingle-unit trucks, three-axle single-unit trucks, four or more axlesingle-unit trucks, four or fewer axle single-trailer trucks, five-axlesingle-trailer trucks, six or more axle single-trailer trucks, five orfewer axle multi-trailer trucks, six-axle multi-trailer trucks and sevenor more axle multi-trailer trucks.

The audio amplitude features may include the amplitude of the audiosignals generated by the vehicle while driving over a road surface and,more particularly, while driving over the road surface at a respectivelocation. The audio frequency features of an example embodiment mayinclude the primary frequency of the audio signals and, in someembodiments, one or more secondary frequencies or the frequency spectrumof the audio signals generated by the vehicle while driving over a roadsurface and, in one embodiment, while driving over the road surface at arespective location. In some embodiments, audio data is created by aplurality of vehicles of the same class while driving over the roadsurface and, more particularly, while driving over the same respectivelocation of the road surface with the audio data created by thedifferent vehicles forming different training examples. In these exampleembodiments, the audio amplitude features may also include one or morestatistical measures of the audio amplitude features of the audio datacreated by the plurality of vehicles of the same class driving over thesame respective location along the road surface. By way of example, thestatistical measures that serve as one or more audio amplitude featuresmay include the mean, the mode, the median, the standard deviation, thevariance, the sigma or the like of the amplitude of the audio signalscreated by the vehicles of the same class driving over the samerespective location along the road surface. Similarly, the statisticalmeasures that serve as one or more audio frequency features may includethe mean, the mode, the median, the standard deviation, the variance,the sigma or the like of the frequency of the audio signals created bythe vehicles of the same class driving over the same respective locationalong the road surface. Additionally or alternatively, the audiofeatures may include one or more audio features that relate or correlateaudio amplitude features and audio frequency features, such as theamplitude of the audio signals at a respective frequency.

The audio features may be represented in various manners including interms of the actual amplitude and the actual frequency of the audiosignals. In an example embodiment, however, the audio features representa change in a respective audio signal relative to a noise floor. Forexample, the audio amplitude features may represent a change inamplitude relative to the noise floor and the audio frequency featuresmay represent a change in frequency relative to the noise floor. Thenoise floor is an average of the audio signals that are captured as avehicle, such as a vehicle of the same class, drives over the roadsurface. In this regard, the noise floor may define an average amplitudeof the audio signals and an average frequency of the audio signalsgenerated by a vehicle of the same class as the vehicle drives over theroad surface. Thus, the audio amplitude features and the audio frequencyfeatures represent deviations from this noise floor, such as may becreated by interaction of the vehicle with a map object, such as a roadseam, a pothole, a transition to a different paving surface or the like.

In some embodiments, a training example also includes informationregarding a map object, such as a map object that is located at thelocation at which the audio data was captured. One example of a mapobject is a bridge and, more particularly, the road seam that is definedbetween the road surface and the bridge surface which typically createsa distinctive audio feature when driven over. Another example of a mapobject is a railroad crossing which similarly creates a distinct audiofeature when a vehicle drives over the railroad crossing. A wide varietyof other map objects may exist including, for example, manhole covers,transitions between different types of road surfaces, such as atransition between a smooth road surface and a grooved road surface, orthe like. As described in more detail below, the map object that isincluded in a training example may be determined or provided in variousmanners. For example, the map object may be defined, either manually orautomatically, by reference to a map database 24 and identification of amap object at the same location or at a location proximate, such aswithin a predefined distance of, the location at which the correspondingaudio data was captured.

The map database 24 may include node data, road segment data or linkdata, point of interest (POI) data, traffic data or the like. The mapdatabase may also include cartographic data, routing data, and/ormaneuvering data. According to some example embodiments, the roadsegment data records may be links or segments representing roads,streets, or paths, as may be used in calculating a route or recordedroute information for determination of one or more personalized routes.The node data may be end points corresponding to the respective links orsegments of road segment data. The road link data and the node data mayrepresent a road network, such as used by vehicles, cars, trucks, buses,motorcycles, and/or other entities. Optionally, the map database maycontain path segment and node data records or other data that mayrepresent pedestrian paths or areas in addition to or instead of thevehicle road record data, for example. The road/link segments and nodescan be associated with attributes, such as geographic coordinates,street names, address ranges, speed limits, turn restrictions atintersections, and other navigation related attributes, as well as POIs,such as fueling stations, hotels, restaurants, museums, stadiums,offices, auto repair shops, buildings, stores, parks, etc. The mapdatabase can include data about the POIs and their respective locationsin the POI records. The map database may include data about places, suchas cities, towns, or other communities, and other geographic featuressuch as bodies of water, mountain ranges, etc. Such place or featuredata can be part of the POI data or can be associated with POIs or POIdata records (such as a data point used for displaying or representing aposition of a city). In addition, the map database can include eventdata (e.g., traffic incidents, construction activities, scheduledevents, unscheduled events, etc.) associated with the POI data recordsor other records of the map database.

The map database 24 may be maintained by a content provider e.g., themap data service provider and may be accessed, for example, by thecontent or service provider processing server. By way of example, themap data service provider can collect geographic data and dynamic datato generate and enhance the map database and dynamic data such astraffic-related data contained therein. There can be different ways usedby the map developer to collect data. These ways can include obtainingdata from other sources, such as municipalities or respective geographicauthorities, such as via global information system databases. Inaddition, the map developer can employ field personnel to travel byvehicle along roads throughout the geographic region to observe featuresand/or record information about them, for example. Also, remote sensing,such as aerial or satellite photography and/or LiDAR, can be used togenerate map geometries directly or through machine learning asdescribed herein. However, the most ubiquitous form of data that may beavailable is vehicle data provided by vehicles, such as mobile device,as they travel the roads throughout a region.

The map database 24 may be a master map database, such as an HD mapdatabase, stored in a format that facilitates updates, maintenance, anddevelopment. For example, the master map database or data in the mastermap database can be in an Oracle spatial format or other spatial format,such as for development or production purposes. The Oracle spatialformat or development/production database can be compiled into adelivery format, such as a geographic data files (GDF) format. The datain the production and/or delivery formats can be compiled or furthercompiled to form geographic database products or databases, which can beused in end user navigation devices or systems.

For example, geographic data may be compiled (such as into a platformspecification format (PSF) format) to organize and/or configure the datafor performing navigation-related functions and/or services, such asroute calculation, route guidance, map display, speed calculation,distance and travel time functions, and other functions, by a navigationdevice, such as by a vehicle represented by mobile device 104, forexample. The navigation-related functions can correspond to vehiclenavigation, pedestrian navigation, or other types of navigation. Thecompilation to produce the end user databases can be performed by aparty or entity separate from the map developer. For example, a customerof the map developer, such as a navigation device developer or other enduser device developer, can perform compilation on a received mapdatabase in a delivery format to produce one or more compiled navigationdatabases.

As mentioned above, the map database 24 may be a master geographicdatabase, but in alternate embodiments, a client side map database mayrepresent a compiled navigation database that may be used in or with enduser devices to provide navigation and/or map-related functions. Forexample, the map database may be used with the mobile device 104 toprovide an end user with navigation features. In such a case, the mapdatabase can be downloaded or stored on the end user device which canaccess the map database through a wireless or wired connection, such asvia a processing server and/or a network, for example.

One example of a set of four training examples is provided below inTable 1 in which a class of vehicle is associated with five featureswith Features 1-4 being different audio features and Feature 5 beinginformation regarding, e.g., the identification of, a map objectassociated with the audio features, such as by being co-located. Of theaudio features, Features 1 and 3 are the frequency and amplitude,respectively, of a first vehicle and Features 2 and 4 are the frequencyand amplitude, respectively, of a second vehicle.

TABLE 1 Vehicle class label Feature 1 Feature 2 Feature 3 Feature 4Feature 5 Truck F_(Trk1) ^(frequency) F_(Trk2) ^(frequency) F_(Trk1)^(amplitude) F_(Trk2) ^(amplitude) bridge Car F_(Car1) ^(frequency)F_(Car2) ^(frequency) F_(Car1) ^(amplitude) F_(Car2) ^(amplitude)Railroad crossing Bus F_(Bus1) ^(frequency) F_(Bus2) ^(frequency)F_(Bus1) ^(amplitude) F_(Bus2) ^(amplitude) bridge Motorcycle F_(MC1)^(frequency) F_(MC2) ^(frequency) F_(MC1) ^(amplitude) F_(MC2)^(amplitude) bridge

In accordance with an example embodiment, the apparatus 10 also includesmeans, such as the processing circuitry 12, the memory 14 or the like,configured to train a machine learning model utilizing the training dataset. See block 32 of FIG. 2. The machine learning model, as trained, isconfigured to predict the class of an unknown vehicle driven over theroad surface based at least in part upon the audio data created by theunknown vehicle. In this regard, the machine learning model is trainedbased upon the plurality of audio features of each training example and,in some embodiments, based upon a map object that is associated with theaudio data of a training example, such as being co-located with and/orlocated within a predefined distance of the location at which the audiofeatures were collected. As noted above, during the training of themachine learning model, the training data sets also include anindication of the class of the vehicle that created the audio data topermit the machine learning model to associate a set of audio featuresand a map object with a respective class of vehicle. Based upon thetraining examples including the plurality of audio features and, in someembodiments, the associated map objects, the machine learning model istrained to predict the corresponding class of vehicle that would havedriven over the road surface in order to create the audio data.

The apparatus 10, such as the processing circuitry 12, may train any ofa variety of machine learning models to predict the class of the vehiclethat created audio data based upon a plurality of audio features and, insome embodiments, a corresponding map object. Examples of machinelearning models that may be trained include a decision tree model, arandom forest model, a neural network, a model that employs logisticregression or the like. In some example embodiments, the apparatus, suchas the processing circuitry, is configured to separately train aplurality of different types of machine learning models utilizing thesame training data including the same plurality of training examples.After having been trained, the apparatus, such as the processingcircuitry, is configured to determine which of the plurality of machinelearning models predicts the class of vehicle that creates the audiodata with the greatest accuracy. The machine learning model that hasbeen trained to most accurately predict the class of vehicle thatcreates the audio data when driven over a road and interacting, forexample, with various map objects is then identified and is thereafterutilized to predict the class of vehicle in response to audio data thathas been collected while a vehicle of an unknown class is driven over aroad surface.

The apparatus 10, such as the processing circuitry 12, is configured todetermine the probability with which a machine learning model, astrained, predicts the class of vehicle that creates audio data that iscaptured while the vehicle drives over a road surface in any of variousmanners. In an example embodiment, one or more additional data sets maybe provided that include audio data as described above and a labelidentifying the class of the vehicle that has driven over the roadsurface to create the audio data of the respective data set. Theapparatus, such as the processing circuitry, then separately utilizeseach of the machine learning models, as trained, to predict the class ofvehicle with the plurality of machine learning models being providedwith the audio data of the one or more additional data sets, but withoutproviding the machine learning models with the class of vehicle thatcreated the audio data. Based upon an analysis of the predictionsprovided by the plurality of machine learning models, the apparatus,such as the processing circuitry, is configured to identify the machinelearning model that has been trained to predict the class of vehiclethat creates the audio data in the most accurate manner. This machinelearning model that is identified as the most accurate may then beutilized to predict a class of vehicle based upon audio data that hasbeen collected without knowledge of the class of vehicle driven over theroad surface during the creation of the audio data.

In some example embodiments, the apparatus 10, such as the processingcircuitry 12, may be configured to evaluate the machine learning model,as trained, to determine the preferred audio features to be processed bythe machine learning model in order to efficiently generate a predictionof the class of vehicle that would have created the audio data includingthe plurality of audio features. Relative to the audio data thatincludes a plurality of audio features created by the interaction of avehicle of a known class with a map object, the apparatus, such as theprocessing circuitry, is configured to provide different combinations ofaudio features to the machine learning model and to determine theaccuracy with which the machine learning model predicts the class of thevehicle driven over the road surface to create the audio data and theefficiency with which the machine learning model predicts the class ofvehicle. In this regard, the efficiency may be defined in variousmanners including the time taken by the machine learning model topredict the class of vehicle, the processing resources required topredict the class of vehicle, the memory resources required to predictthe class of vehicle, the number of computations required to predict theclass of vehicle, etc. Based upon the efficiency and the accuracy withwhich the machine learning model predicts the class of vehicle inresponse to different combinations of the audio features, the preferredset of audio features that allows the machine learning model to providea reasonably accurate prediction of the class of vehicle in an efficientmanner may be identified.

In some example embodiments, a balance or trade off between the accuracywith which the class of vehicle is predicted and the efficiency withwhich the machine learning model predicts the class of vehicle isconsidered. For example, a first combination of audio features mayproduce the most accurate prediction of the class of vehicle, but asecond combination of audio features may produce a prediction of theclass of vehicle that is only slightly less accurate, but that issignificantly more efficient in terms of its prediction. Thus, thesecond combination of audio features that provide for sufficient, eventhough not the greatest, accuracy, but does so in a very efficientmanner may be identified by the apparatus 10, such as the processingcircuitry 12, as the preferred combination of audio features to beprovided to the machine learning model in order to predict the class ofvehicle in subsequent instances in which the audio data that has beencaptured includes at least the preferred combination of audio features.

As described above, the training examples each include audio data and,more particularly, a plurality of audio features and a label thatidentifies the class of vehicle that created the audio data and, in someembodiments, information that identifies a map object associated withthe audio data, such as being co-located with or within a predefineddistance of the location at which the audio data was captured. In someexample embodiments, at least some of the training examples may alsoinclude one or more additional types of information or data. Forexample, the audio data created by a vehicle of a certain class drivingover a road surface may vary based upon the speed of the vehicle. Forexample, a passenger car driving across a railroad crossing at 5 mph andat 25 mph may create two different and distinctive sets of audiofeatures. Thus, in an example embodiment, at least some of the trainingexamples may also include the speed at which the vehicle was driven atthe time at which the audio data was captured. By being trained topredict the class of vehicle based not only upon the audio data and, insome embodiments, the map object, but also, in some embodiments, basedupon the speed of the vehicle, the machine learning model, as trained,may thereafter be utilized to predict the class of vehicle with enhancedaccuracy and/or efficiency in instances in which the audio data that hasbeen collected is also associated with the speed of the vehicle at thetime at which the audio data was collected.

The training examples may be constructed in a variety of differentmanners. In one embodiment that is provided with reference to FIG. 3 byway of example, but not of limitation, however, the apparatus 10includes means, such as the processing circuitry 12, the memory 14 orthe like, for constructing the respective training example byidentifying a map object with reference to a map database, such asstored by the memory or a database 24 with which the apparatus, such asthe processing circuitry, is in communication via the communicationinterface 16. See block 40. In this example embodiment, the apparatus,such as the processing circuitry, is configured to identify the audiodata that was captured at a location proximate the map object byreference to an audio database, such as provided by the memory or by adatabase, with which the apparatus, such as the processing circuitry, isin communication via the communication interface. See block 42 of FIG.3. In this regard, the audio database provided by the map database and,as a result, may be stored by the same database that stores the mapdatabase. Alternatively, the audio database may be distinct from the mapdatabase and stored by the same or a different database than that whichstores the map database. The audio data is associated with a location atwhich the audio data was previously captured and the audio data that isidentified in conjunction with a respective map object has a locationthat is proximate the map object. Thus, the audio data that isidentified may be associated with the same location as the location ofthe map object or with a location that is within a predefined distanceof the location of the map object. In addition to including theplurality of audio features, the audio data that is identified may notonly be associated with a location at which the audio data was captured,but may also be associated with a class of vehicle that created theaudio data and, in some embodiments, a speed with which the vehicle wastraveling at the time at which the audio data was created. As shown inblock 44, the apparatus of an example embodiment also includes means,such as the processing circuitry or the like, for identifying the classof vehicle that created the audio data. Based upon the audio data thatis identified, a training example is constructed for each instance ofaudio data that is identified to be proximate the respective map objectwith each training example including the audio features of a respectiveinstance of the audio data, a corresponding map object and a labelidentifying the class of vehicle that created the audio data andoptionally an indication of the speed at which the vehicle was travelingwhen the audio data was created.

Although the map database and the audio database may be constructed invarious manners, the audio database may be included within or otherwiseassociated with the map database. In one example embodiment, the audiodatabase is defined by an audio layer that is associated with the mapdatabase. In this example embodiment, the audio layer includes the audiodata and the related information, such as the class of vehicle andoptionally the speed of the vehicle, and is associated with a locationthat is correlated to the locations of the various map objectsmaintained by the map database. While all of the audio data may beincluded in a single audio layer in some embodiments, the audio databaseof other example embodiments includes a plurality of audio layers, eachof which is correlated to the map database including the plurality ofmap objects based upon the respective locations of the audio data in theaudio layer and the map objects in the map database. In these otherexample embodiments in which the audio database includes a plurality ofaudio layers, each audio layer may include the audio data created by adifferent respective class of vehicle. Thus, one audio layer would beassociated with audio data created by passenger automobiles, while adifferent audio layer would include audio data created by a motorcycle.

The audio layer including the audio data created by a respective classof vehicle may, in some embodiments, be further divided into a pluralityof sub-layers, each of which includes the audio data created by therespective class of vehicle while traveling at a different respectivespeed or a different respective range of speeds. By way of example, theaudio layer that includes audio data created by a passenger automobilemay include a plurality of sub-layers including a first audio sub-layerthat includes the audio data captured for a passenger automobiletraveling at less than 20 mph, a second audio sub-layer including theaudio data captured for a passenger automobile traveling between 20 and30 mph, a third audio sub-layer including the audio data captured for apassenger automobile traveling at a speed between 30 and 40 mph, etc.

Further details are provided by U.S. patent application Ser. No.16/699,907 filed Dec. 2, 2019 and U.S. patent application Ser. No.16/699,921 filed Dec. 2, 2019, the entire contents of each of which areexpressly incorporated by reference herein in their entirety.

In order train the machine learning model, the apparatus 10, such as theprocessing circuitry 12, the memory 14 or the like, of an exampleembodiment may therefore be configured to identify a map object from themap database and to then identify and access the audio data associatedwith the selected map object, such as by the audio data that isco-located with the map object or that is within a predefined distanceof the map object. With reference to FIG. 4 by way of example, a roadsegment 50 including two map objects, namely, railroad tracks 52 and aseam 54 being two different types of road surfaces, is depicted. In themap database 24, each map object is associated with a location along theroad segment as represented by an X in FIG. 4. In this example, theinstances of audio data proximate a respective map object areidentified. In FIG. 4, the instances of audio data are represented bydots 56 and those instances of audio data that are within a predefineddistance of the location of a respective map object (as represented bythe outline 58) are identified to be proximate the respective mapobject. The apparatus, such as the processing circuitry, of this exampleembodiment is then configured to construct training examples based uponthe map object and associated audio data and to then train the machinelearning model utilizing the training examples such that the resultingmachine learning model is capable of predicting the class of vehiclethat creates audio data, preferably in an accurate and efficient manner.

Once the machine learning model has been trained, the machine learningmodel may be utilized to predict the class of vehicle traveling over aroad surface based upon audio data that is created by the vehicle eventhough the class of vehicle is not known in advance. As shown in block60 of FIG. 5, the apparatus 10 of this example embodiment includesmeans, such as the processing circuitry 12, the memory 14, thecommunication interface 16 or the like, for obtaining audio data createdby a vehicle while driving over a road surface. The audio data includesa plurality of audio features including one or more audio amplitudefeatures or one or more audio frequency features or a combination ofaudio amplitude features and audio frequency features, such as describedabove. The audio data may be collected by one or more audio sensors onboard the vehicle. For example, the audio sensors may be associated withone or more respective wheels of the vehicle that is driving over theroad surface. The apparatus, such as the processing circuitry, may beconfigured to obtain the audio data that has been collected by the audiosensor(s) and to utilize the trained machine learning model to predictthe class of vehicle that has created the audio data. This predictionmay occur in real time or near real time while the vehicle continues totravel over the road network or at some later time after the vehicle hasdriven over the road surface. In an instance in which the predictionoccurs at a later point in time, the audio data may be collected andstored by the memory or a database 24 with which the apparatus, such asthe processing circuitry, the communication interface or the like, is incommunication and may be subsequently accessed, such as in a batchprocess, in order to permit the classes of a plurality of vehicles thathave traveled over the road surface, such as during some predefinedwindow of time, to be analyzed and/or to permit a distribution of theclasses of vehicle that travel over a respective road surface, such asduring a predefined period of time, to be constructed.

As shown in block 64 of FIG. 5, the apparatus 10 of this exampleembodiment also includes means, such as the processing circuitry 12, thememory 14, the communication interface 16 or the like, for providing theaudio data including one or more audio amplitude features or one or moreaudio frequency features or a combination of one or more audio amplitudefeatures and one or more audio frequency features, to the machinelearning model, as has been trained, such as described above withrespect to FIG. 2. In this example embodiment, the apparatus furtherincludes means, such as the processing circuitry or the like, forpredicting the class of vehicle that created the audio data utilizingthe machine learning model. See block 66 of FIG. 5.

In an example embodiment in which a preferred combination of audiofeatures has been identified to permit the machine learning model topredict the class of vehicle with sufficient accuracy and efficiency,the apparatus 10, such as the processing circuitry 12 or the like, isconfigured to determine whether the audio data that has been obtainedincludes at least the preferred combination of audio features. In aninstance in which the audio data that is obtained does include thepreferred combination of audio features, the apparatus, such as theprocessing circuitry, the memory 14 or the like, may be configured toprovide the preferred combination of audio features to the machinelearning model without providing any additional audio features that areincluded within the audio data that has been collected and obtained.Thus, the apparatus, such as the processing circuitry of this exampleembodiment may be configured to predict the class of vehicle thatcreated the audio data in an accurate manner, but to do so efficiently,based upon the preferred combination of audio features. However, in aninstance in which the audio data that is obtained does not include thepreferred combination of audio features (such as by failing to includeone or more of the preferred combination of audio features), theapparatus, such as the processing circuitry, the memory or the like, maybe configured to provide all of the audio features that have beenobtained to the machine learning model including, for example, one ormore audio features that are not included within the preferredcombination of audio features.

In some example embodiments, the apparatus 10, such as the processingcircuitry 12, is configured to provide the machine learning model notonly with the audio data, but also with information regarding one ormore map objects. In this regard, the audio data that has been collectedmay have been created at the same location or proximate to, such aswithin a predefined distance of, a map object. Indeed, the interactionof the vehicle with the map object may have caused the audio data to becreated. Thus, the apparatus of this example embodiment includes means,such as the processing circuitry, the memory 14, the communicationinterface 16 or the like, for identifying one or more map objects andfor providing information regarding the one or more map objects to themachine learning model in conjunction with the audio data. See blocks 62and 64 of FIG. 5. In this example embodiment, the apparatus, such as theprocessing circuitry, is configured to predict the class of vehicleutilizing the machine learning model and based upon both the one or moremap objects and the audio data.

The one or more map objects may be identified in various manners. In anexample embodiment, the audio data that is collected by one or moreaudio sensors on board the vehicle is associated with the location ofthe vehicle at the time at which the audio data is captured. Forexample, the location may be provided by a location sensor, such as aGPS sensor, on board or otherwise configured to monitor the location ofthe vehicle. In this example embodiment, the apparatus 10, such as theprocessing circuitry 12, the memory 14, the communication interface 16or the like, is configured to obtain the audio data that is collectedand, based upon the location of the vehicle at which the audio data wascollected, to also identify the one or more map objects at or proximate,that is, within a predefined distance of, that same location, such as byreference to a map database. For example, the audio data that isobtained may include a corresponding location at which the audio datawas captured. The apparatus, such as the processing circuitry, thememory, the communication interface or the like, is then configured toaccess a map database that includes one or more map objects, each ofwhich is associated with a respective location. Based upon the locationof the audio data that has been obtained, the apparatus, such as theprocessing circuitry, is configured to identify one or more map objectsthat are associated with the same location or with a location that isproximate to, such as by being within a predefined distance of, thelocation at which the audio data was captured.

One example of four different instances of audio data is depicted belowin relation to Table 2. Each instance of audio data includes four audiofeatures and an associated map object (feature 5). In this example, thefirst audio feature is the primary frequency, the second audio featureis the mean of the frequencies, the third audio frequency is theamplitude and the fourth audio frequency is the amplitude of the primaryfrequency.

TABLE 2 Feature 1 Feature 2 Feature 3 Feature 4 Feature 5 F_(Trk1)^(frequency) F_(mean) ^(frequency) F_(Trk1) ^(amplitude) F_(primfreq)^(amplitude) bridge F_(Car1) ^(frequency) F_(mean) ^(frequency) F_(Car1)^(amplitude) F_(primfreq) ^(amplitude) Railroad crossing F_(Bus1)^(frequency) F_(mean) ^(frequency) F_(Bus1) ^(amplitude) F_(primfreq)^(amplitude) bridge F_(MC1) ^(frequency) F_(mean) ^(frequency) F_(MC1)^(amplitude) F_(primfreq) ^(amplitude) bridge

Although audio data may be collected over a period of time, such ascontinuously, while a vehicle drives over a road surface, the audio datathat is obtained and is utilized to predict the class of vehicle may beonly a subset of all of the audio data that is collected, with thesubset of audio data being that audio data that is more distinctive and,as a result, more likely to be indicative of interaction of the vehiclewith a map object as opposed to audio data that is merely representativeof background or normal road noise. In this regard and as describedabove, the audio features, such as the one or more audio amplitudefeatures or the one or more audio frequency features or a combination ofone or more audio amplitude features and one or more audio frequencyfeatures, that represent a change in amplitude and/or a change infrequency relative to a noise floor are identified, such as by theapparatus 10, such as the processing circuitry 12.

The noise floor represents the average background noise for a vehicledriving over the road surface. As the average background noise generallyvaries depending upon the speed with which the vehicle is traveling, theaudio data that is collected and is subsequently obtained in accordancewith an example embodiment may be associated with the speed of thevehicle at the time at which the audio data was collected. In thisregard, the speed of the vehicle may be provided by a speed sensorcarried by the vehicle and saved in association with the audio data thatis collected. As such, different noise floors may be defined fordifferent speeds at which a vehicle is traveling at the time at whichthe audio data is collected. However, as the class of vehicle that istraveling over the road surface is unknown during the utilization of thetrained machine learning model, the noise floor is typically notdependent upon the class of vehicle.

Thus, the apparatus of an example embodiment that includes differentnoise floors for different speeds or different ranges of speeds includesmeans, such as the processing circuitry 12 or the like, for receiving anindication of the speed of the vehicle. In this regard, the audio datathat is obtained may be associated with the speed of the vehicle at thetime at which the audio data was collected such that the speed of thevehicle is received in conjunction with obtaining the audio data. Inthis example embodiment, the apparatus also includes means, such as theprocessing circuitry, the memory 14 or the like, for determining thenoise floor based at least in part upon the speed of the vehicle. Inthis regard, the noise floor may have been defined in advance for eachof a plurality of different vehicle speeds or different ranges ofvehicle speeds with the noise floor that has been defined for the speedor range of speeds at which the vehicle was traveling while collectingthe audio data being identified for use in conjunction with the audiodata. Alternatively, the noise floor may be defined by the apparatus,such as the processing circuitry, based upon the audio data collected bythe vehicle under analysis so as to be representative of the averageroad noise collected as audio data while the vehicle drives over theroad network.

Once the noise floor has been determined, such as based upon the speedof the vehicle, the apparatus 10 of this example embodiment, such as theprocessing circuitry 12, is configured to identify the audio featuresthat differ from the noise floor by a predefined amount or by apredefined percentage and may then utilize only the audio data thatdiffers from the noise floor by the predefined amount or the predefinedpercentage in order to determine the class of the vehicle that createdthe audio data. Additionally or alternatively, the apparatus, such asthe processing circuitry, the memory 14 or the like, may be configuredto only store those instances of the audio data that differ from thenoise floor by the predefined amount or the predefined percentage,thereby increasing the efficiency with which the audio data may bestored and retrieved while still retaining the audio data that isrepresentative of the most significant deviations from background roadnoise and that are therefore the most useful with respect to theidentification of the class of vehicle creating the audio data.

As described above, the map objects may be stored in a map database thatidentifies the different types of map objects and the location at whichthe map objects are located. Additionally, an audio database may bemaintained, such as by the memory 14 or a database 24 with which theapparatus 10, such as the processing circuitry 12, is in communication,such as via the communication interface 16, with the audio databaseincluding the audio data that has been captured. In an exampleembodiment, the audio database may be an audio layer including the audiodata and correlated, such as based upon the location at which the audiodata was collected, to the map database and, more particularly, to thelocations of the plurality of map objects included within the mapdatabase. As also described above, the audio layer of an exampleembodiment may include a plurality of sub-layers, each of which isassociated with the audio data that has been collected for a vehicletraveling at a different speed or within a different range of speeds.Thus, the apparatus of an example embodiment, such as the processingcircuitry, the memory, the communication interface or the like, may beconfigured to obtain the audio data by reference to an audio layer andmay obtain the corresponding map object(s) by reference to the mapdatabase.

In an example embodiment, the apparatus 10 also includes means, such asthe processing circuitry 12, configured to determine the predictionconfidence associated with the class of vehicle that has been predictedby the machine learning model. In this regard, the prediction confidenceis an indication of the correctness of the prediction of the class ofvehicle and, in one embodiment, may range from zero in which there is noconfidence in the prediction to one in which there is absoluteconfidence in the prediction. The prediction confidence may bedetermined in various manners. For example, the machine learning modelas implemented by the apparatus, such as the processing circuitry, of anexample embodiment is configured to determine the prediction confidencebased upon the alignment of the data that has been collected foranalysis by the machine learning model with the data of the trainingdata sets. More alignment results in increased prediction confidence,while less alignment results in less prediction confidence. In terms ofalignment, the type of data and/or the value of the data may beconsidered. Data that has been collected that is of the same type andthat has a value within a predefined range of the value of the same typeof data from the training data sets provides for increased alignment andcorrespondingly increased predication confidence. Conversely, data thathas been collected that is of a different type or that has a value thatis outside of a predefined range of the value of the same type of datafrom the training data sets provides for reduced alignment andcorrespondingly reduced predication confidence. By way of example, in aninstance in which the collected data includes the same five types ofdata with the same range of values as the training data, the predictionconfidence will be greater than an instance in which the collected dataonly has three of the five types of data as the training data, even ifthe data remains within the same range of values.

In this example embodiment, in an instance in which the confidence inthe prediction by the machine learning model is considered high, such asin an instance in which the prediction confidence exceeds a firstpredefined value, such as by being greater than 0.75, the predictionprovided by the machine learning model as to the class of the vehiclemay be accepted. In contrast, in an instance in which the confidence inthe prediction by the machine learning model is considered low, such asin an instance in which the prediction confidence is less than a secondpredefined value, such as by being less than 0.5, the predictionprovided by the machine learning model may be rejected and the class ofthe vehicle based upon the audio data may remain unknown orindeterminate notwithstanding the prediction provided by the machinelearning model due to the relatively low prediction confidence.

In an intermediate instance in which the prediction confidence isbetween the first and second predefined values, the prediction of thevehicle class provided by the machine learning model may be consideredto be possibly correct but without sufficient confidence to rely uponthe prediction without further substantiation. In this exampleembodiment, the class of the vehicle that has been predicted may besubjected to further verification. Although this further verificationmay be provided in various manners, the verification of class of vehiclemay, in some embodiments, be provided by manual or automated analysis ofone or more images of the vehicle obtained by traffic cameras or othertechniques for capturing and analyzing images of a road network and thetraffic thereover. In an instance in which the prediction by the machinelearning model is verified, the predicted class of vehicle may then beaccepted. However, if the class of vehicle that has been predicted bythe machine learning model is unable to be verified, the predicted classof vehicle may be rejected and the class of the vehicle may beconsidered unknown or indeterminate notwithstanding the predictionprovided by the machine learning model. By utilizing predictionconfidence in this example embodiment, the predictions of vehicle classprovided by the machine learning model and subsequently accepted basedupon the prediction confidence may then be utilized with greaterconfidence in the results.

Based upon the prediction of the class of vehicle that created the audiodata, the method, apparatus 10 and computer program product of anexample embodiment may provide a variety of advantages. For example,information regarding the number of vehicles of the different classesthat travel over a road segment, such as a distribution of the differentclasses of vehicles that travel over a road segment, may be collectedand provided, such as to federal, state, local or other governmental orregulatory authorities responsible for the construction and maintenanceof the road network. This information may facilitate the subsequentconstruction of road networks to ensure that the resulting road networksare designed to support the different classes of vehicle that areanticipated to travel thereover. Additionally, this information mayfacilitate the maintenance of roads by permitting those roads havinggreatest use by heavier vehicles that may create more damage to theroads to be reviewed for maintenance on a more regular basis.

The information regarding the class of vehicle that has created audionoise may also be utilized in a more specific manner, such as to providetargeted information or advertisements to a respective vehicle basedupon the class of vehicle as predicted by the machine learning model. Inthis regard, the targeted advertising or other information may beprovided via a computing device, such as a navigation, mapping orinfotainment system, carried by the vehicle or via a mobile devicecarried by the driver or other occupant of the vehicle. By way ofexample, in an instance in which the vehicle is identified to be a truckand trailer combination having four or more axles, information may beprovided regarding upcoming truck stops or other areas that are designedto facilitate ingress and egress by a tractor trailer. Alternatively, inan instance in which the vehicle is identified to be a motorcycle,targeted advertising relating to nearby motorcycle rallies or motorcyclestores or information regarding routes that have been frequented in thepast by motorcyclists, such as scenic routes, may be provided in orderto improve the user experience.

FIGS. 2, 3 and 5 illustrate flowcharts depicting a method according toan example embodiment of the present invention. It will be understoodthat each block of the flowcharts and combination of blocks in theflowcharts may be implemented by various means, such as hardware,firmware, processor, circuitry, and/or other communication devicesassociated with execution of software including one or more computerprogram instructions. For example, one or more of the proceduresdescribed above may be embodied by computer program instructions. Inthis regard, the computer program instructions which embody theprocedures described above may be stored by a memory device 14 of anapparatus 10 employing an embodiment of the present invention andexecuted by the processing circuitry 12. As will be appreciated, anysuch computer program instructions may be loaded onto a computer orother programmable apparatus (for example, hardware) to produce amachine, such that the resulting computer or other programmableapparatus implements the functions specified in the flowchart blocks.These computer program instructions may also be stored in acomputer-readable memory that may direct a computer or otherprogrammable apparatus to function in a particular manner, such that theinstructions stored in the computer-readable memory produce an articleof manufacture the execution of which implements the function specifiedin the flowchart blocks. The computer program instructions may also beloaded onto a computer or other programmable apparatus to cause a seriesof operations to be performed on the computer or other programmableapparatus to produce a computer-implemented process such that theinstructions which execute on the computer or other programmableapparatus provide operations for implementing the functions specified inthe flowchart blocks.

Accordingly, blocks of the flowcharts support combinations of means forperforming the specified functions and combinations of operations forperforming the specified functions for performing the specifiedfunctions. It will also be understood that one or more blocks of theflowcharts, and combinations of blocks in the flowcharts, can beimplemented by special purpose hardware-based computer systems whichperform the specified functions, or combinations of special purposehardware and computer instructions.

Many modifications and other embodiments of the inventions set forthherein will come to mind to one skilled in the art to which theseinventions pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the inventions are not to be limited to the specificembodiments disclosed and that modifications and other embodiments areintended to be included within the scope of the appended claims.Furthermore, in some embodiments, additional optional operations may beincluded. Modifications, additions, or amplifications to the operationsabove may be performed in any order and in any combination.

Moreover, although the foregoing descriptions and the associateddrawings describe example embodiments in the context of certain examplecombinations of elements and/or functions, it should be appreciated thatdifferent combinations of elements and/or functions may be provided byalternative embodiments without departing from the scope of the appendedclaims. In this regard, for example, different combinations of elementsand/or functions than those explicitly described above are alsocontemplated as may be set forth in some of the appended claims.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purposes of limitation.

What is claimed is:
 1. An apparatus configured to predict a class of avehicle based at least in part on audio data, the apparatus comprisingat least one processor and at least one memory storing computer programcode, the at least one memory and the computer program code configuredto, with the processor, cause the apparatus to at least: obtain theaudio data created by the vehicle while driving over a road surface,wherein the audio data comprises one or more audio amplitude features;provide the audio data including the one or more audio amplitudefeatures to a machine learning model; predict the class of the vehiclethat created the audio data utilizing the machine learning model;identify one or more map objects; and provide information regarding theone or more map objects to the machine learning model, wherein the atleast one memory and the computer program code are configured to, withthe processor, cause the apparatus to predict the class of the vehicleby predicting the class of the vehicle utilizing the machine learningmodel and based upon the one or more map objects and the audio data. 2.The apparatus according to claim 1, wherein the at least one memory andthe computer program code are further configured to, with the processor,cause the apparatus to receive an indication of a location of thevehicle, wherein the at least one memory and the computer program codeare configured to, with the processor, cause the apparatus to obtain theaudio data by obtaining the audio data captured proximate the locationof the vehicle, and wherein the at least one memory and the computerprogram code are configured to, with the processor, cause the apparatusto identify the one or more map objects by identifying the one or moremap objects proximate the location of the vehicle.
 3. The apparatusaccording to claim 1, wherein the at least one memory and the computerprogram code are configured to, with the processor, cause the apparatusto obtain the audio data by capturing the audio data associated with oneor more respective wheels of the vehicle while driving over the roadsurface and associating the location of the vehicle during creation ofthe audio data therewith.
 4. The apparatus according to claim 1, whereinthe one or more audio amplitude features represent a change in amplituderelative to a noise floor.
 5. The apparatus according to claim 4,wherein the at least one memory and the computer program code arefurther configured to, with the processor, cause the apparatus to:receive an indication of a speed of the vehicle; and determine the noisefloor based at least in part upon the speed of the vehicle.
 6. A methodfor predicting a class of a vehicle based at least in part on audiodata, the method comprising: obtaining the audio data created by thevehicle while driving over a road surface, wherein the audio datacomprises one or more audio frequency features; providing the audio dataincluding the one or more audio frequency features to a machine learningmodel; predicting the class of the vehicle that created the audio datautilizing the machine learning model; identifying one or more mapobjects; and providing information regarding the one or more map objectsto the machine learning model.
 7. The method according to claim 6,further comprising receiving an indication of a location of the vehicle,wherein obtaining the audio data comprises obtaining the audio datacaptured proximate the location of the vehicle, and wherein obtainingthe one or more map objects comprises identifying the one or more mapobjects proximate the location of the vehicle.
 8. The method accordingto claim 6, wherein obtaining the audio data comprises capturing theaudio data associated with one or more respective wheels of the vehiclewhile driving over the road surface and associating the location of thevehicle during creation of the audio data therewith.
 9. The methodaccording to claim 6, wherein the one or more audio frequency featuresrepresent a change in frequency relative to the noise floor.
 10. Themethod according to claim 9, further comprising: receiving an indicationof a speed of the vehicle; and determining the noise floor based atleast in part upon the speed of the vehicle.
 11. An apparatus configuredto train a machine learning model to predict a class of an unknownvehicle driving over a road surface, the apparatus comprising at leastone processor and at least one memory storing computer program code, theat least one memory and the computer program code configured to, withthe processor, cause the apparatus to at least: provide a trainingdataset comprising a plurality of training examples, wherein arespective training example comprises audio data created by a vehicle ofa known class while driving over a road surface, wherein the audio datacomprises one or more audio features selected from one or more audioamplitude features and one or more audio frequency features, and whereinthe respective training example also comprises the class of the vehiclethat created the audio data; and train the machine learning modelutilizing the training dataset such that the machine learning model, astrained, is configured to predict the class of the unknown vehicledriven over the road surface based at least in part upon the audio datacreated by the unknown vehicle wherein the respective training examplefurther comprises a map object.
 12. The apparatus according to claim 11,wherein the at least one memory and the computer program code areconfigured to, with the processor, cause the apparatus to provide thetraining dataset by constructing the respective training example byidentifying the map object by reference to a map database andidentifying the audio data proximate the map object by reference to anaudio database, wherein the audio data is associated with a locationproximate the map object at which the audio data was previouslycaptured.
 13. The apparatus according to claim 12, wherein the at leastone memory and the computer program code are further configured to, withthe processor, cause the apparatus to construct the respective trainingexample by identifying the class of the vehicle that created the audiodata while driving over the road surface at the location proximate themap object.
 14. The apparatus according to claim 12, wherein the atleast one memory and the computer program code are configured to, withthe processor, cause the apparatus to identify the audio data byidentifying the audio data proximate the map object from an audio layerof the map database, wherein the audio layer defines a plurality ofinstances of audio data in association with respective locations.